Automating Vehicles by Deep Reinforcement Learning using Task Separation with Hill Climbing

Within the context of autonomous driving a model-based reinforcement learning algorithm is proposed for the design of neural network-parameterized controllers. Classical model-based control methods, which include sampling- and lattice-based algorithms and model predictive control, suffer from the trade-off between model complexity and computational burden required for the online solution of expensive optimization or search problems at every short sampling time. To circumvent this trade-off, a 2-step procedure is motivated: first learning of a controller during offline training based on an arbitrarily complicated mathematical system model, before online fast feedforward evaluation of the trained controller. The contribution of this paper is the proposition of a simple gradient-free and model-based algorithm for deep reinforcement learning using task separation with hill climbing (TSHC). In particular, (i) simultaneous training on separate deterministic tasks with the purpose of encoding many motion primitives in a neural network, and (ii) the employment of maximally sparse rewards in combination with virtual velocity constraints (VVCs) in setpoint proximity are advocated.Comment: 10 pages, 6 figures, 1 tabl

Structure of the FoxM1 DNA-recognition domain bound to a promoter sequence

FoxM1 is a member of the Forkhead family of transcription factors and is implicated in inducing cell proliferation and some forms of tumorigenesis. It binds promoter regions with a preference for tandem repeats of a consensus ‘TAAACA’ recognition sequence. The affinity of the isolated FoxM1 DNA-binding domain for this site is in the micromolar range, lower than observed for other Forkhead proteins. To explain these FoxM1 features, we determined the crystal structure of its DNA-binding domain in complex with a tandem recognition sequence. FoxM1 adopts the winged-helix fold, typical of the Forkhead family. Neither ‘wing’ of the fold however, makes significant contacts with the DNA, while the second, C-terminal, wing adopts an unusual ordered conformation across the back of the molecule. The lack of standard DNA–‘wing’ interactions may be a reason for FoxM1’s relatively low affinity. The role of the ‘wings’ is possibly undertaken by other FoxM1 regions outside the DBD, that could interact with the target DNA directly or mediate interactions with other binding partners. Finally, we were unable to show a clear preference for tandem consensus site recognition in DNA-binding, transcription activation or bioinformatics analysis; FoxM1's moniker, ‘Trident’, is not supported by our data

FOXM1 binds directly to non-consensus sequences in the human genome.

BACKGROUND: The Forkhead (FKH) transcription factor FOXM1 is a key regulator of the cell cycle and is overexpressed in most types of cancer. FOXM1, similar to other FKH factors, binds to a canonical FKH motif in vitro. However, genome-wide mapping studies in different cell lines have shown a lack of enrichment of the FKH motif, suggesting an alternative mode of chromatin recruitment. We have investigated the role of direct versus indirect DNA binding in FOXM1 recruitment by performing ChIP-seq with wild-type and DNA binding deficient FOXM1. RESULTS: An in vitro fluorescence polarization assay identified point mutations in the DNA binding domain of FOXM1 that inhibit binding to a FKH consensus sequence. Cell lines expressing either wild-type or DNA binding deficient GFP-tagged FOXM1 were used for genome-wide mapping studies comparing the distribution of the DNA binding deficient protein to the wild-type. This shows that interaction of the FOXM1 DNA binding domain with target DNA is essential for recruitment. Moreover, analysis of the protein interactome of wild-type versus DNA binding deficient FOXM1 shows that the reduced recruitment is not due to inhibition of protein-protein interactions. CONCLUSIONS: A functional DNA binding domain is essential for FOXM1 chromatin recruitment. Even in FOXM1 mutants with almost complete loss of binding, the protein-protein interactions and pattern of phosphorylation are largely unaffected. These results strongly support a model whereby FOXM1 is specifically recruited to chromatin through co-factor interactions by binding directly to non-canonical DNA sequences.We would like to acknowledge the Genomics and bioinformatics core at the CRUK Research Institute for the Illumina sequencing and the Proteomics core for the LC/MS-MS protein analysis for the RIME experiments. We acknowledge the support from The University of Cambridge and Cancer Research UK. The Balasubramanian Laboratory is supported by core funding from Cancer Research UK (C14303/A17197). SB is a Wellcome Trust Principle Investigator.This is the final version of the article. It first appeared from BioMed Central via http://dx.doi.org/10.1186/s13059-015-0696-

Molecular basis of FIR-mediated c-myc transcriptional control

The far upstream element (FUSE) regulatory system promotes a peak in the concentration of c-Myc during cell cycle. First, the FBP transcriptional activator binds to the FUSE DNA element upstream of the c-myc promoter. Then, FBP recruits its specific repressor (FIR), which acts as an on/off transcriptional switch. Here we describe the molecular basis of FIR recruitment, showing that the tandem RNA recognition motifs of FIR provide a platform for independent FUSE DNA and FBP protein binding and explaining the structural basis of the reversibility of the FBP-FIR interaction. We also show that the physical coupling between FBP and FIR is modulated by a flexible linker positioned sequentially to the recruiting element. Our data explain how the FUSE system precisely regulates c-myc transcription and suggest that a small change in FBP-FIR affinity leads to a substantial effect on c-Myc concentration.MRC Grant-in-aid U11757455

Warped Riemannian metrics for location-scale models

The present paper shows that warped Riemannian metrics, a class of Riemannian metrics which play a prominent role in Riemannian geometry, are also of fundamental importance in information geometry. Precisely, the paper features a new theorem, which states that the Rao-Fisher information metric of any location-scale model, defined on a Riemannian manifold, is a warped Riemannian metric, whenever this model is invariant under the action of some Lie group. This theorem is a valuable tool in finding the expression of the Rao-Fisher information metric of location-scale models defined on high-dimensional Riemannian manifolds. Indeed, a warped Riemannian metric is fully determined by only two functions of a single variable, irrespective of the dimension of the underlying Riemannian manifold. Starting from this theorem, several original contributions are made. The expression of the Rao-Fisher information metric of the Riemannian Gaussian model is provided, for the first time in the literature. A generalised definition of the Mahalanobis distance is introduced, which is applicable to any location-scale model defined on a Riemannian manifold. The solution of the geodesic equation is obtained, for any Rao-Fisher information metric defined in terms of warped Riemannian metrics. Finally, using a mixture of analytical and numerical computations, it is shown that the parameter space of the von Mises-Fisher model of $n$-dimensional directional data, when equipped with its Rao-Fisher information metric, becomes a Hadamard manifold, a simply-connected complete Riemannian manifold of negative sectional curvature, for $n = 2,\ldots,8$. Hopefully, in upcoming work, this will be proved for any value of $n$.Comment: first version, before submissio

FOXM1 drives proximal tubule proliferation during repair from acute ischemic kidney injury

The proximal tubule has a remarkable capacity for repair after acute injury, but the cellular lineage and molecular mechanisms underlying this repair response are incompletely understood. Here, we developed a Kim1-GFPCreERt2 knockin mouse line (Kim1-GCE) in order to perform genetic lineage tracing of dedifferentiated cells while measuring the cellular transcriptome of proximal tubule during repair. Acutely injured genetically labeled clones coexpressed KIM1, VIMENTIN, SOX9, and KI67, indicating a dedifferentiated and proliferative state. Clonal analysis revealed clonal expansion of Kim1+ cells, indicating that acutely injured, dedifferentiated proximal tubule cells, rather than fixed tubular progenitor cells, account for repair. Translational profiling during injury and repair revealed signatures of both successful and unsuccessful maladaptive repair. The transcription factor Foxm1 was induced early in injury, was required for epithelial proliferation in vitro, and was dependent on epidermal growth factor receptor (EGFR) stimulation. In conclusion, dedifferentiated proximal tubule cells effect proximal tubule repair, and we reveal an EGFR/FOXM1-dependent signaling pathway that drives proliferative repair after injury

Implementing EM and Viterbi algorithms for Hidden Markov Model in linear memory

<p>Abstract</p> <p>Background</p> <p>The Baum-Welch learning procedure for Hidden Markov Models (HMMs) provides a powerful tool for tailoring HMM topologies to data for use in knowledge discovery and clustering. A linear memory procedure recently proposed by <it>Miklós, I. and Meyer, I.M. </it>describes a memory sparse version of the Baum-Welch algorithm with modifications to the original probabilistic table topologies to make memory use independent of sequence length (and linearly dependent on state number). The original description of the technique has some errors that we amend. We then compare the corrected implementation on a variety of data sets with conventional and checkpointing implementations.</p> <p>Results</p> <p>We provide a correct recurrence relation for the emission parameter estimate and extend it to parameter estimates of the Normal distribution. To accelerate estimation of the prior state probabilities, and decrease memory use, we reverse the originally proposed forward sweep. We describe different scaling strategies necessary in all real implementations of the algorithm to prevent underflow. In this paper we also describe our approach to a linear memory implementation of the Viterbi decoding algorithm (with linearity in the sequence length, while memory use is approximately independent of state number). We demonstrate the use of the linear memory implementation on an extended Duration Hidden Markov Model (DHMM) and on an HMM with a spike detection topology. Comparing the various implementations of the Baum-Welch procedure we find that the checkpointing algorithm produces the best overall tradeoff between memory use and speed. In cases where sequence length is very large (for Baum-Welch), or state number is very large (for Viterbi), the linear memory methods outlined may offer some utility.</p> <p>Conclusion</p> <p>Our performance-optimized Java implementations of Baum-Welch algorithm are available at <url>http://logos.cs.uno.edu/~achurban</url>. The described method and implementations will aid sequence alignment, gene structure prediction, HMM profile training, nanopore ionic flow blockades analysis and many other domains that require efficient HMM training with EM.</p

A cross-cultural comparison of student learning patterns in higher education

Marambe, K. N., Vermunt, J. D., & Boshuizen, H. P. A. (2012). A cross-cultural comparison of student learning patterns in higher education. Higher Education, 64(3), 299-316. doi:10.1007/s10734-011-9494-zThe aim of this study was to compare student learning patterns in higher education across different cultures. A meta-analysis was performed on three large-scale studies that had used the same research instrument: the Inventory of learning Styles (ILS). The studies were conducted in the two Asian countries Sri Lanka and Indonesia and the European country The Netherlands. Students reported use of learning strategies, metacognitive strategies, conceptions of learning and learning orientations were compared in two ways: by analyses of variance of students' mean scale scores on ILS scales, as well as by comparing the factor structures of the ILS-scales between the three studies. Results showed most differences in student learning patterns between Asian and European students. However, many differences were identified between students from the two Asian countries as well. The Asian learner turned out to be a myth. Moreover, Sri Lankan students made the least use of memorising strategies of all groups. That Asian learners would have a propensity for rote learning turned out to be a myth as well. Some patterns of learning turned out to be universal and occurred in all groups, other patterns were found only among the Asian or the European students. The findings are discussed in terms of learning environment and culture as explanatory factors. Practical implications for student mobility in an international context are derived

Participatory instructional redesign by students and teachers in secondary education: effects on perceptions of instruction

Könings, K. D., Brand-Gruwel, S., & Van Merriënboer, J. J. G. (2011). Participatory instructional redesign by students and teachers in secondary education: effects on perceptions of instruction. Instructional Science, 39(5), 737–762.Students’ perceptions of instruction are important because they direct the learning of students. The fact that teachers have only limited knowledge of these perceptions is likely to threaten the effectiveness of learning, because congruence between interpretations of an instructional intervention is necesarry for its optimal use. This study examines participatory design as a strategy for taking student perceptions into account in instructional re/design. Participatory design meetings of groups of teachers and seven co-designing students in a secondary education setting identified changes to improve the regular education process. The results on changes in student perceptions, perceived-desired discrepancy, and teacher-student disagreement showed some improvement for the co-designers but, unexpectedly, limited or even negative effects for the non-co-designing students. Possible causes are discussed. Participatory design seems to have potential for improving education, but further research is needed